Automatic Smoothing and Variable Selection via Regularization

This thesis focuses on developing computational methods and the general theory of automatic smoothing and variable selection via regularization. Methods of regularization are a commonly used technique to get stable solution to ill-posed problems such as nonparametric regression and classification. In recent years, methods of regularization have also been successfully introduced to address a classical problem in statistics, variable selection. Smoothing parameters are introduced in regularization to balance the tradeoff between the goodness-of-fit to the data such as the log likelihood, and the prior knowledge on the unknown such as the degree of smoothness. A successful regularization method requires an objective way to tune the smoothing parameter involved. In this thesis, we demonstrate the use of methods of regularizations and different approaches for tuning parameter selection in three different settings, namely nonparametric Gaussian heteroscedastic regression, nonparametric Poisson regression, and variable selection for normal linear models. We start with the nonparametric heteroscedastic regression where both the conditional mean and variance are assumed unknown. A penalized log likelihood method, doubly penalized log likelihood estimate, is proposed to estimate both the conditional